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Abstract — We study noisy computation in randomly 
generated k-ary Boolean formulas. We establish bounds on 
the noise level above which the results of computation by 
random formulas are not reliable. This bound is saturated 
by formulas constructed from a single majority-like gates. 
We show that these gates can be used to compute any 
Boolean function reliably below the noise bound. 

Index Terms — Random Boolean formulas, e-noise, reli- 
able computation. 

I. Introduction 

One of computation models for a Boolean function 
f : {—1,1}" — > { — 1,1} is a Boolean circuit or formula 
0"). A circuit is a directed acyclic graph in which nodes 
of in-degree zero are either the Boolean constants or 
variables, nodes of in-degree k > 1 are logical gates, 
computing Boolean functions of k arguments, and nodes 
of out-degree zero correspond to the circuit outputs. If 
a circuit has only a single output and the output of each 
gate is used as an input to at most one gate then this 
circuit is called a formula. In circuits, as in any other 
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model of computation, the computational complexity and 
effects of noise are important questions [0. 

The circuit complexity of a Boolean function is the 
minimum number of gates (circuit size) or the minimum 
t/e/??/[] of a circuit, constructed from a particular set of 
gates, which computes this function. However, to find 
a circuit representation of a Boolean function with a 
bounded size or depth is a difficult problem [1]. One ap- 
proach to this problem is to study complexity of typical 
Boolean functions computed by random formulas Q. 

The two most studied methods of generating random 
formulas use random tree generation and a growth pro- 
cess as their core procedures. In the first method, a rooted 
fc-ary tree is sampled from the uniform distribution of 
all rooted fc-ary trees; the leaves of this random tree are 
then labelled by reference to the Boolean variables and 
internal nodes are labelled by the Boolean gates. This 
method was used to investigate the complexity of typical 
functions computed by random AND/OR formulas [4|, 
1 5 1, |6| and allowed to obtain a close relation between 
the probability P[f] of a random formula to compute a 
Boolean function / and its size (complexity). However, it 
seems that this probability distribution is biased towards 
very low complexity functions 0. 

'The depth of a circuit is the number of gates on the longest path 
from an input node to the output node 
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The second method uses the following growth process: 
Firstly, one defines the initial probability distribution 
Po [f ] over the set F° of simple Boolean functions of N 
variables. Secondly, and in further steps, the functions 
chosen from the distributions P t [f] defined in previ- 
ous steps are combined by Boolean gates: F t+1 = 
Mfi,...,^ G F*fbrj = 1,2,...,*}. This pro- 
cess can be seen as a growth of fc-ary balanced trees and 
was first used by Valiant to obtain an upper bound on 
the size of monotone formulas computing the majority 
function [7|. Savicky recently showed for one of these 
processes, for Po[f] that is uniform on some set of 
Boolean functions F° and under very broad conditions 
on a, the probability P t [f] tends to the uniform distri- 
bution over all Boolean functions of N variables when 
t —¥ oo lISl. The convergence rates of the Savicky's 
process and its variants with different gates and initial 
conditions were studied in 0. 

Another important question in the circuit theory is 
a reliable computations of Boolean functions in the 
presence of noise. One of the first to study the effect 
of noise in computing systems was von Neumann who 
attempted to explain the robustness of biologically- 
inspired computing circuits J5]. His model represented 
neural activities by a circuit (or formula) composed of 
e-noisy Boolean gates. The e-noisy gate is designed to 
compute a Boolean function a : { — 1, l} fe — > { — 1,1}, 
but for each input a E { — l,l} fc there is an error 
probability e such that a (a) —> —a(a). To simplify 
the analysis, error-probability is taken to be independent 
for each gate in the circuit. Clearly, a noisy circuit 
(e > 0) cannot perform any given computation in a 
deterministic manner: for any circuit-input there is a non- 
vanishing probability that the circuit will produces the 
wrong output. The maximum of this error probability S 
over all circuit-inputs determines the reliability of the 



circuit. In his paper, von Neumann showed that reliable 
computation {5 < 1/2) is possible for a sufficiently small 
e (9j and demonstrated how reliability of a Boolean noisy 
circuit can be improved by using constructions based 
only on e-noisy gates. 

There had been little development in the analysis 
of noisy computing systems until the seminal work of 
Pippenger iflOl who addressed the problem from an in- 
formation theory point of view. He showed that if a noisy 
fc-ary formula is used to compute a Boolean function / 
with the error probability 5 < 1/2, then (i) there is an 
upper bound for the gate-error e(fc) which is strictly less 
than 1/2 and (ii) there is a lower bound for the formula- 
depth d(k, e, S) > d, where d is the depth of a noiseless 
formula computing /. In comparison to its noiseless 
counterpart, a noisy formula that computes reliably has 
greater depth due to the presence of restitution-gates, 
implying longer computation times IflOl . 

A number of papers have followed and extended 
Pippenger's results. For instance, similar results were 
derived for circuits by Feder lUTI . who also improved 
the bounds obtained by Pippenger for formulas. The 
exact noise thresholds for fc-ary Boolean formulae were 
later determined for odd fc iflZl . lfP3l and for formulas 
constructed from 2-input NAND gates |fl4l: the latter 
was recently suggested as the exact noise threshold for 
general 2-input gate formulas |15|. 

Results derived for noisy Boolean formulas in fPH . 
Ifl3l rely on a specific construction which uses e-noisy 
majority gates. The noiseless variant of this gate per- 
forms the majority-vote function^ sgn^*^ Sj] on the 
binary inputs Sj € { — 1,1} and naturally the number of 
these inputs fc is odd. In contrast to previous work, in 

2 We use the definition sgn[z] = 1 for x > 0, sgn[x] = — 1 for 
x < and sgn[0] =0 throughout this paper. 
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1 -1 



ffc= 1 



-1 



y 



many realizations of noise leads us to the equation 



■a^k-*- -1 1 



1 =f 



Fig. 1. Noisy growth process, i) Boolean functions fi,...,fj. 
(represented by binary strings of length 2") are sampled randomly and 
independently from the distribution Pt[f]- ii) These functions are then 
used to compute a new Boolean function f via the gate a. At each step 
of this computation noise (represented by the binary string £) inverts 
the output of a (this operation is represented by the X symbol) with 
probability e. In this figure the first and the last bits of the function 
f (in red) are inverted by noise. Repeating operations i) and ii) many 
times gives rise to an ensemble described by the distribution Pt+i[f]. 



this paper we concentrate on the possibility of reliable 
computation in randomly generated Boolean formulas. 
As a first step towards this goal, we study the effects 
of e-noise on the formulas generated in the Savicky's 
growth process. 



II. Noisy growth processes and main results 

Let us introduce noise into the formulas generated by 
Savicky's growth process. In order to do this we note 
that the noiseless case, as described in the Introduction, 
can be also seen as a computation, performed by gate a, 
of a new Boolean function f from k Boolean functions 
fi,...,ffc. These functions, represented by binary ±1 
strings (or vectors) of length 2 , are drawn randomly 
and independently from the same distribution [8 |. How- 
ever, each computation at the gate a may be corrupted 
by noise that inverts the result of this computation with 
probability e (see Figure [TJ. Averaging the process over 



P*+i[f] = 



{fj}j=i 



(i) 



n 



a /3f*a(f;,...,f£) 



AA2co8h/9a(fI,...,f£) 
which gives us the probability of a Boolean function f 
being computed by the noisy formulas of depth t + 1. 
Here for convenience we have introduced the inverse 
"temperature" parameter f3 — 1/T which is related to 
the noise parameter e via the equality tanh /3 = 1 — 2e. 
The limits f3 — > O/oo correspond to completely ran- 
dom/deterministic cases. 

Without noise ((3 — > oo) the equation (Q3 reduces to 

2 N 

x*[[6 [?;<*&,...,{},)], 
»=i 

where we use 5[x;y) to denote Kronecker delta. Equa- 
tion (fJJ was studied in the original Savicky's work JS] 
and subsequent studies [3| where the stationary distribu- 
tion Poo[f] = linii_>oo Pj[ f ] of the noiseless process 
(O was studied with the initial conditions Po [ f ] = 
TFtrr Y^geF" IL=i <5p;g1 f ° r different initial sets F u of 
simple Boolean functions (constants, identities, etc.) and 
different gates a. Depending on these parameters the 
stationary distribution is either concentrated on a single 
function, i.e. Poo[f] = Ili=i ^ [ f ; g z J or on some set 
of functions F, i.e. P^[f] - ^ £ geF ITi=i *[f*; g*]. 
There are also cases when for t — > oo the distributions 
Pt[f] and P t+ i[f] are distinct. 

Our main contribution to these studies is the following 
result for the noisy process ([T). 



Theorem 2.1: For a balanced gat 



a the stationary 



3 The gate is balanced when it has an equal number of +l's and 
— l's in its output. 
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distribution Poo[f] = -^w is a stable and unique solution 
of the process (fl]i when e > e(k) = ^—j^-, where b(k) = 

{2 fc - 1 /M (fc !7) 1 / 2 ); 2 fe -7(fc-i)( (fe ^ /2 )}, with k > 3, 

for k odd and even respectively. 

Proof: In order to show this, we first use the result, 
derived in Appendix [A] that the distribution P t+ i[f] 
can be represented via its moments m s (t + 1) = 
Pt+i [f] YiieS f l > wnere S is a subset of the set 
[2"] = {!,..., 2^}, and P t+1 [f] is given by 



L[f] 



1 

2~2* 



1+ J] m s (t + l)[]f ! 



(3) 



SC[2»] iSS 

where the 7i-th moment is governed by the equation 

mV-^Ct + l) (4) 

= tanh"(/3)^n{ 

{f?}J =1 ^ 



2™ 



SCI 



,4) 



with I = {ix, . . . , i n }. Thus the n-th moment at t + 1 is 
a function of only the rt-th and lower order moments at 
t. 

Let us assume now that tanh(/3) < b(k) then by 
the Lemma 14.11 with n = 1, the first moments of 
the distribution P t+ i[f] are vanishing as t —> oo. But 
then, by applying the same lemma to the order n > 2 
moments, we conclude that all moments are vanishing 
as t —> oo. ■ 

In addition to its direct interpretation that above e(k) 
(see Figure [U the noisy process ([T) is ergodic and has 
only one stationary solution, the result of Theorem 12.11 
also has consequences for computation in noisy random 
formulas. A feature of noisy formulas, which is essential 
for reliable computation, is their greater depth due to the 
presence of correcting e-noisy gates 1 10 1. This correction 




100 



Fig. 2. Upper bound for reliable computation by noisy fc-ary random 
formulas. 



operation can be seen as a procedure which reduces the 
entropy, but in our case of very deep (t — > oo) random 
formulas the entropy is at its maximum when e > e(k). 
Thus any computation, even as simple as computing 
identity function, can not be performed reliably in this 
regime. 

For odd k our result for the bound e(fc) is exactly 
equal to the exact threshold for reliable computation by 
general fc-ary formulas |[T2l . lfT3l . It is not clear however 
if this threshold is also exact, i.e. any Boolean function 
can be computed for e g (0, e(fc)) with the error 5 < 
1/2, for randomly generated formulas. For even k > 
2 this threshold is not known, but our result suggests 
that for balanced gates a it can not exceed the bound 
e(k) of Theorem 12.11 Furthermore as k — > oo the e(k) 
approaches 1/2 as 1/2— e(fc) = 0(l/y/k), this is follows 
from the Stirling's approximation of b(k), which is in 
agreement with the bound computed in [16| for general 
formulas. 

The results of Lemma 13.11 which are used in the 
proof of Theorem 12. II can be also exploited to show that 
reliable computation in randomly generated formulas is 
possible. This can be shown as follows. Suppose that 
gate a in the process (03 is the same as the one stud- 
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ied in Lemma 13.11 Assume that the initial distribution 
Po[f] is such that the stationary distribution Poo[f] of 
the noiseless process (O is concentrated on only one 
Boolean function, i.e. all formulas compute the same 
function. This implies that for any input / e {— 1, 1}^ 
all formulas simultaneously provide an output of +1 or 
— 1. Then in the presence of noise the average formula 
errors in its output with the probability (1 — m/(oo))/2, 
where m/(oo) is the stationary solution of equation (0 
corresponding to this input. From the analysis in Lemma 
B.ll follows that the maximum of this error over all inputs 
is 5 = (1 — m(oo))/2 and is bounded away from 1/2 
when e < e(k). Furthermore, the output error can be 
reduced by decreasing e or by increasing k. Thus in this 
regime any Boolean function can be computed with any 
desired accuracy. 

III. Computation of the lower bound values 

In this section we compute the values of lower bounds 
appearing in Theorem 12.11 In order to do this we 
choose a balanced gate x(er) from the set of gates 



sgn 



Ek 
.1=1 u i 



J2j=i a i = 7( a )> where 7(0-) e 



{-1, 1} is such that J^a 1 ELi < j . 







7(0") = 0, and 



consider the first moments mi(t) — J^t Pt[ f ] ^- These 
are governed by the equations 



m(t + l) = F x (m(t)) 



(5) 



tanh(/3) sgn[2^ - k] 

1=0 

m(t) 



1 



1 - m(<) 



-1 k—t 



Lemma 3.1: For k > 3 the function F x (m), defined 
in the equation (0, has the following properties: i) if 

tanh/3 < b(k) then m > F x (m) for m £ (0,1] and 
F x (m) > m for m e [-1,0); ii) if tanh/3 > b{k) then 



3 m* 7^ such that m* = F x (m*), where b(k) is defined 
in the Theorem 12. II 

Proof: This lemma follows from the equalities 
F x (±l) = ±tanh^,F x (0) = (this can be shown by 
direct substitution) and the fact that F x (m) is a strictly 
increasing function, which is also convex and concave on 
the intervals (—1, 0) and (0, 1), respectively (to show this 
we study properties of F x (m) in the Appendix 151. Then 

i) is true because -n^| m =o < 1 when tanh/3 < b(k) and 

ii) is true because of -3^|m=o > 1 when tanh/3 > b(k). 



IV. Moments of P t [ f ] 

Let us consider equation for an n-th moment m. 
Assuming that all lower order moments vanish allows us 
to write this equation in a very simple form 



m(t+l) = F a (m(t)) 



(6) 



tanh*G8) £ [] 



2" 



l + m(t) jjerj 



For a balanced gate a the point m = is a stationary 
solution of the above equation and has the following 
property. 

Lemma 4.1: The point m = is a stable and unique 
solution of © when tanh™(/3) < b(k). 

Proof: In order to prove this we first show that 



tanh(/3)"- 1 F x (m) 

= tanh-Cfl E IT F 

k n 

xsgn Y.Y\ a 
3=1 i=i 



(7) 



1 + mJ]^} 
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where F x {m) is defined in (0. This can be shown by a 
direct calculation as follows 



tanh"(/3)^n4 



x sen 



eik 

3=1 «=1 



1 



i=l 



(8) 



{^}i=i 



men 



x sgn 



k 

E-) 

3=1 



tanh(/3)"- 1 F x (m). 



In the above the first equality was obtained by applying 
transformation erj — > Y\i=i a j an< ^ me l ast equality 
followed from comparing this result with the right hand 
side of equation (0. 

Next, for a balanced gate a we compute the difference 
A(m) = tanh" _1 (^)F x (m) - F Q (m) in Appendix O 
The result of this computation is that A(m) > and 
A(m) < on the intervals me [0, 1) and m€ (—1, 0], re- 
spectively, from which the bounds tanh™~ 1 (/3)F x (m) > 
F Q (m) and tanh ,l_1 (^)F x (m) < F Q (m) on the same 
intervals follow. The behaviour of tanh ra_1 (/3)F x (m) 
with respect to the inverse temperature f3 is the same 
as of F x (m), which we described in Lemma [XT1 but 
with the tanh(/3) being replaced by the tanh™(/3). 



V. Conclusion 

The paper extends previous work ifPJl . ifPJI on the 
reliability of computation in Boolean formulas and 
generation of random Boolean functions (8), 0, by 
investigating the properties of formulas constructed by 
a random growth process whereby computing elements, 
primarily /c-ary balanced gates, are subject to e-noise. 

We show that the noisy growth process is ergodic 
above the noise bound e(fc) and hence the formulas 



generated by it are unreliable. We also show that formu- 
las constructed from majority-like gates, which saturate 
this bound, can be used for computing any Boolean 
function when e < e(fc). Our earlier work, which uses 
methods of non-equilibrium statistical physics, suggests 
that the same noise bound also applies to the noisy feed- 
forward [17 1 and recurrent Boolean networks [fl"8l . 

The current analysis is restricted to reliable computa- 
tion in a growth process that uses only balanced gate^ 
and produces (without noise) only one Boolean function; 
but we envisage that it can be extended to study more 
general scenarios of non-balanced gates and a richer 
distributions of Boolean functions J3|. 

Appendix A 
Moment representation of P t [ f ] 

The probability distribution P t [ f ] can be represented 
via its moments. In order to find this representation we 
can use the identity 5[f ; f] = 1 to write P t [f] = 
J2t <S[f; f]P*[f]- Then, because S[x;y] = i(l + xy) for 
x, y € { — 1, 1}, we obtain 

P t [f] = ^P t [f][]-(l + ff) (9) 



\ f SC[2 N ]i€S ) 

wv+ E - s (')lm> 

\ SC[2«] ies J 



where m s (t) = P*[f] Tii^s ^ are me moments of 

Let us now derive equation governing evolution of 
the n-th moment m 11 ' 12 '"'' 1 " (i). This can be obtained by 
multiplying both sides of equation (Q~|i by the monomial 

4 The results of this paper can be easily extended to the distributions 
over balanced gates 1181 . 
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riigif 1 ' where I = {ii, . . . ,i n }, and taking the sums 
over f as follows 



E p *+i[ f ]Il f 



(10) 



iei 



: EEII{ p '[ f i]} 

f ft}i=i 

2 „/3f»a(f?,...,«) 

e . T[f 

= tanh^CS) X;iI{P*[^,...,^]} 

{f)}i =1 

fj) 

iei 

:m 11 — 4 "(t+l), 



where in the above we have used the property 
tanh(— x) — — tanh(x). Finally, using the moment 
representation ((9} in the above, we obtain 



fc 

= tanh"(/3)^n 

{f|}J =1 



Appendix B 
Derivation of F x and analysis of its 
properties 

Here we first derive the function F x (m) then we study 
its properties. Let us first compute the sum 



fc r 

En 

{f 3 }j=i 



[l + f,- m" 


| sgn 


k 


2 











(12) 



+1 



E 



E f , = o 

7 = 1 



7(fi,.-.,ffc) 

i + m -| 



1 — m 



1 (fe-E?=i 



x< sgn 



E f * 

7 = 1 



E f , = ° 

7 = 1 



7(fi,---,ffc) 



E 

£=0 



1 + m 



1 — m 



sgn[2^ - k] , 



in the definition ( fTOb for the specific choice of a = \- 
This result leads to the function F x (m) used in equation 
©. 

We are interested in how the function F x (m) behaves 
on the interval m e [—1,1] and how this behaviour is 
affected by the parameter tanh/3. In order to find this 
out we first rewrite F x (m) as follows 



(13) 



tanh(/3) 

k 



1 + m 



><E 

£=0 

x {l[2i - k > 0] - 1[2£ - fc < 0]} , 



1 — m 



(ID but 



E 

£=0 



1 + m 



1 — m 



k-e 



(14) 



2™ 



i+E^on^ 



SCI 



ies 



1 + m 



f r 



1 - m 



k-e 



= E 

x {1[2£ - k > 0] + 1[2I - fc < 0]} 
= 1 
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SO 



F x (m) 

= tanh(/3)[l -2^ 



(15) 



e=o 



1 + m 



1 — m 



k—l 



-l[fc = (mod 2)] 



(fc/2 



1 + m 



1 — m 



fe/2 ' 



where 



k = l[k = (mod 2)] (fc/2- 1) 
+ l[k = (mod l)](fc- l)/2. 

Now we use the above representation of F x (m) to 
compute 



(16) 



tanh(/3) 



(k-£) 


"1 + 


m 




"1 


2 








"1 + m" 




"1 


— m 


2 




2 



1 — m 
"2" 

fc-C s 



k-l-l 



+ tanh(/3)^l[fc = (mod 2)] Q 



1 + m 



1 — m 



fe/2-i 



tanh^-^fc + l) 



1 + m 



1 — m 



fc-fc-l 



+ tanh(/3)Jl[fc = 0(mod 2)] ( * 



1 + m 



1 — m 



fc/2-l 



So, using definition of fe, we obtain 



d 

dm 



F x (m) 



^iik+m 



k + l 



1 + m 



for fc odd and 



_d 

dm 



tanh(/3) 



fc 
fc/2 

1 + m 



1 — m 



fc 
4 

1 — m 



2 

(fc-l)/2 



(17) 



(18) 



fc/2-l 



for fe even. 

Thus ^F x (m) > on for all m 6 (—1,1) and hence 
F x (m) is a strictly increasing function. Furthermore, the 
function F x (m) at the point m = changes its slope 
from JLF x (m)| m=0 < 1 to £.F x (m)\ m=Q > 1 at 

tanh(/3) = 2 k - 1 /k ''' ~ ' 

for fe odd and 

tanh(/3) = 2 k - 2 / 



(fc-l)/2, 

(fe-1) 



fe- 2 
(fc-2)/2, 

for fc even. 

Let us now compute the second derivative of F x (m). 
Differentiating equations (TTTI l and ( fl~8l with respect to 



m gives us 
d 2 

dm 2 xv ; 



-mtanh(j8) (19) 

/ k \ fk + l\ (fe-1 
X ^ + l)/2j 

1 + ml I" 1 — m 



4 

(fc-l)/2-l 



for fe odd and 

d 2 



dm 2 



F x (m) 



-mtanh(/3) (20) 

/ fc ^ fk\ (fc-2) 
X U/2 



1 + m 



4 

1 — m 



fc/2-2 



for fc even. We note that both are of the form 
^jF x (m) = -mG(m), where G(m) > for all 
m e (—1,1). Thus the function F x (m) is strictly 
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convex and concave on the intervals (—1,0) and (0, 1) 
respectively. 



Appendix C 
Computation of A(m) and analysis of its 

PROPERTIES 

Let us first define the average ("') a \ m = 

E{<x-} U k j= i ^ [1 + m ]ir=i °j] (■■■) and shorthand 
notations {1 + [cr], l_[cr], lo[u]} for the indicator func- 

tions {iEj =1 m=i°i > o]' i Ej= 1 nr=i^ < 

0], n"=i a j = 0]}- Then the rescaled difference 

A(m)/4tanh™(/3) can be computed as follows 



to obtain 



A(m) 



4tanh n (/3) 

'1 + m 



(23) 



xl+ [a]l 



2" 

1+m 



1 — m 
2™ 



E 



1 — m 



n«w,...,<.)=-i 

.2=1 

iSj=inr=i^i 



1 — m 



1 + m 
Xl_ [a] 1 



A(m) 



n«K,...,4)=+i 
i - i 

-^l„Hl]aW,..,<) . 



4tanh™(/3) 



sgn 



A: n 



En-] 

i=l i=l 



(21) 



(r|m 



Now because a is a balanced gate we have the following 
identity 



~i !+[*]-!- [a] 



(24) 



(1+ [a] + 1- [a] + 1 M)IJ«M..-.,4) 

i=i / 

n 

n«(4,...,4)=-i 

i=i 

IJa((r*,...,(7i)=+l 



-i-Wi 



1 " 

-1 [S\l[a{ai,...,4) 



= J2 (l+M+l-W+loM ) 
x (l f[a(4,...,4) = +l 

V L=i 

n 

-l n>(4,...,4)=-i 

j=i 

/ n 

= E 2ioMn«M»--.4) 



tj\r 



In the above we can use the identity 



1 [a] 1 



i+ M l 



k 

TT- 

j=i 



1+m 



t +£j=in? = i'j 



[]a(4,...,4) = +l 

i=l 
n 

n«(4,---,4)=-i 



t=i 



(22) 



0. 



1 — m 



fc-E, fc =1 n? =1 -' 



Adding the above representation of zero to the terms 
inside the curly brackets in equation (l23l leads to the 
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final result 



A(m) 

= 2tanh"(/3) 



(25) 



1+m 



1 — m 




1 + m 



2™ 



iE? =1 n? =1 ^i \ 



1 — m 



x 1+ [a] 1 



f]a(ai,...,4) = -l 




1 — m 



1+m 



iE? =1 n? =1 ^-i \ 



X 1_ [cr] 1 



Y[a{a\,...,ai) = +1 



From the above it is clear that A(m) > for m€ [0, 1) 
and A(m) < for me (-1, 0]. 
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